@Jens-G It seems the exporter is sending values through the display formatter, which doubles backslashes for human-readable SELECT output, before handing them to the CSV writer. Claude suggests just stop using the display formatter for CSV export of text in copyutil.py. What would you think of that approach?

def format_value(self, val, cqltype):
    if val is None or val == EMPTY:
        return format_value_default(self.nullval, colormap=NO_COLOR_MAP)

    # Text-like values: pass through unmodified. The csv.writer will apply
    # CSV-level escaping using the dialect's escapechar. Running these
    # through the display formatter would double-escape backslashes, see
    # CASSANDRA-21131.
    if cqltype.type_name in ('text', 'varchar', 'ascii'):
        return val
    ...

TBH if it works I'm fine with either approach 👍 🚀

Hi @Jens-G , if you are busy I can add the unit tests for this and you can review that — just let me know if that works. ?

@Jens-G the PR needs additional work. Based on our discuss of RFC 4180, proper text values don't require formatting. In format_value():

if cqltype.type_name in ('text', 'varchar', 'ascii'):
    return val if val.isprintable() else None

that may eliminate the need to have a new argument for escape_backslash

Thanks @bschoening! I worked through the scalar fast-path. It's a clean simplification for text/varchar/ascii, but I don't think it lets us drop the escape_backslash argument — here's the case it misses.

The ticket also covers list<text>, set<text>, map<text,text>, tuples and UDTs. Those aren't serialized as scalars: format_value_list/_set/_map/_utype format each element by routing it back through format_value_text (with quote=True). A cqltype.type_name in ('text','varchar','ascii') check in format_value() never sees them, so their elements still get their backslashes doubled. Since collection elements always pass through format_value_text, the no-double-escape signal has to live on that function regardless of what we do at the scalar level — so the argument can't be fully removed without dropping collection support.

I've added TestExportFormatValue in this PR to pin this down. The collection cases serialize list/set/map/tuple<text> through the export path and assert single backslashes; they fail (doubled) if the escape_backslash=False change is reverted — which is effectively what a scalar-only check does to collections. (On import there's no compensation either: the elements are CQL string literals, and dequote_value only un-doubles single quotes, not backslashes.)

On return val if val.isprintable() else None: with the parameter approach it's moot, but for the record else None would silently null the whole value for any text containing a newline or tab (it becomes an empty field). I understand the intent of keeping control characters out of the CSV — but that's separate from the backslash doubling, and format_value_text rewrites control chars via UNICODE_CONTROLCHARS_RE either way, so I'd keep it out of this ticket and handle it in the UNICODE_CONTROLCHARS_RE follow-up I mentioned.

Happy to add a scalar fast-path on top of the parameter if you'd like it for readability — as an addition rather than a replacement. WDYT?

@arvindKandpal-ksolves thanks a lot for offering to help! I've gone ahead and added the unit tests in this PR (TestExportFormatValue in test_copyutil.py), so this one is covered. Much appreciated. 🙏

@Jens-G Based on what we learned with CASSANDRA-21381, we should broaden the scope here to handle all TEXTDATA properly.

COPY TO should allow roundtrip all TEXTDATA. Backslash is just another character in the %x2D-7E range - a perfectly ordinary TEXTDATA character with no special status so escape_backslash as an option isn't needed. The only escape production is DQUOTE.

1. format_value: agreed, let's fast-path it — skip formatting for printable text and sanitize only the non-printable cases..

2. control characters: agree that else None is too aggressive. Text containing TAB, CR, LF should be enclosed in double quotes (the CSV escaped form), in a sanitize field function. The remaining C0/C1 controls are outside the scope of RFC 4180; rather than support round-tripping them, I'd suggest a one-way, percent-encoded translation (%00, %1B, …) — non-reversible by design, and less easily confused with Python's own backslash escaping.

3. CSV escaping implies that COPY FROM would revert DQUOTE escapes

4. collections: agreed -- list, set, map<text,text> need the same sanitization and should be in scope here.

BTW, sineemore/csv-test-data has test fixtures we could cherry-pick for unit tests.

Relevant RFC 4180 grammer:

field = (escaped / non-escaped)

escaped = DQUOTE *(TEXTDATA / COMMA / CR / LF / 2DQUOTE) DQUOTE

non-escaped = *TEXTDATA

TEXTDATA = %x20-21 / %x23-2B / %x2D-7E

DQUOTE = %x22